Video Polyp Segmentation: A Deep Learning Perspective

We present the first comprehensive video polyp segmentation (VPS) study in deep learning era. Over years, developments VPS are not moving forward with ease due to lack of large-scale fine-grained annotations. To address this issue, we introduce a high-quality frame-by-frame annotated dataset, named SUN-SEG, which contains 158,690 colonoscopy frames from well-known SUN-database. provide additional annotations diverse types, i.e., attribute, object mask, boundary, scribble, and polygon. Second, design simple but efficient baseline, dubbed PNS+, consisting global encoder, local normalized self-attention (NS) blocks. The encoders receive an anchor frame multiple successive extract long-term short-term spatial-temporal representations, then progressively updated by two NS Extensive experiments show that PNS+ achieves best performance real-time inference speed (170fps), making it promising solution for task. Third, extensively evaluate 13 representative polyp/object models on our SUN-SEG dataset attribute-based comparisons. Finally, discuss several open issues suggest possible research directions community.